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DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments, see Remarks pages 9-10, filed 12/08/2008, with respect 
to the rejection(s) of claim(s) 1-27 and 29 under 35 U.S.C. 102(a) have been fully 
considered and are persuasive. Therefore, the rejection has been withdrawn. 
However, upon further consideration, a new ground(s) of rejection is made in view of 35 
U.S.C. 102(e) with reference to Toklu et al. Figure 2C. 

2. Re claim 29, the Applicant contends that the sequence of frames in Toklu is not 
from the same scene/area of interest. However, the Examiner respectfully disagrees. 
The new rejection below refers to Fig. 2C of Toklu wherein camera motion parameters 
are not taken into account, thus eliminating the influence of camera motion in key frame 
extraction. Additionally, Toklu discloses that several key frames may come from a 
single video segment (Toklu: column 6, lines 30-32). 



Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or 
composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, 
subject to the conditions and requirements of this title. 

5. Claims 1-27 and 29 are rejected under 35 U.S.C. 101 as not falling within 
one of the four statutory categories of invention. 

Supreme Court precedent [Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker 
v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalkv. Benson, 409 U.S. 63, 70 (1972); 
Cochrane v. Deener, 94 U.S. 780, 787-88 (1876)] and recent Federal Circuit decisions 
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[In re Bilski, 88 USPQ2d 1385 (Fed. Cir. 2008)] indicate that a statutory "process" under 
35 U.S.C. 101 must (1) be tied to another statutory category (such as a particular 
apparatus), or (2) transform underlying subject matter (such as an article or material) to 
a different state or thing. While the instant claim(s) recite a series of steps or acts to be 
performed, the claim(s) neither transform underlying subject matter nor positively tie to 
another statutory category that accomplishes the claimed method steps, and therefore 
do not qualify as a statutory process. For example, the methods recited do not specify 
any physical components for carrying out each of the steps described. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

4. Claims 1, 2, 14-17, 23, 24, and 29 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Tolku et al. (US 6549643 B1). 

Re claim 1, Toklu discloses a method of identifying a plurality of key video 
frames in a sequence of image frames, each of said sequence of image frames 
containing a plurality of pixels, each of said plurality of pixels corresponding to a 
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corresponding point of an area based on which said sequence of image frames are 
generated, said method comprising: determining a rate of change of visual content of 
each current frame from a corresponding reference frame, each of said current frame 
and said reference frame being comprised in said sequence of image frames (Toklu: 
Fig. 2C, step 229, smooth temporal color histogram activity curve), wherein said rate of 
change represents a difference of a first value and a second value, said first value 
representing a change of visual content of a current frame compared to a first frame, 
said second value representing a change of visual content of said first frame compared 
to a second frame, wherein said second frame is a reference frame for said first frame 
and said first frame is a reference frame for said current frame (Toklu: Fig. 2C, step 228; 
Equation 1 1 , temporal color histogram activity curve used to generate binary color 
histogram activity curve); and selecting said current frame as a corresponding one of a 
set of potential video frames if said rate exceeds a corresponding first threshold value, 
wherein said plurality of key video frames are selected from said set of potential video 
frames (Toklu: Equation 11), wherein said determining and said selecting are repeated 
for each of said sequence of image frames as said current frame to form said set of 
potential video frames (Toklu: column 11, lines 36-38), wherein each of the respective 
first frame and the respective second frame are different for different ones of the current 
frame in said sequence of image frames (Toklu: column 1 1 , lines 2-10). 

Re claim 2, Tolku discloses a majority of the features of claim 2, as discussed 
above in claim 1, and Toklu additionally discloses determining a displacement 
magnitude of each moved pixel of said current frame compared to the position in said 
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first frame and of said first frame compared to the position in said second frame (Toklu: 
Fig. 2C, step 228); and computing a first representative magnitude of said displacement 
magnitude for said moved pixels of said current frame compared to said first frame, and 
a second representative magnitude of said displacement magnitude for said moved 
pixels of said first frame compared to said second frame (Toklu: Fig. 2C, step 229, 
smooth temporal color histogram activity curve), wherein said first value and said 
second value respectively equal said first representative magnitude and said second 
representative magnitude such that said rate is computed as a difference of said first 
representative magnitude and said second representative magnitude (Toklu: Equation 
11). 

Re claim 14, Toklu discloses that the corresponding first frame and the 
corresponding second frame are selected at a same respective relative position in 
comparison to the position of the current frame such that each of the respective first 
frame and the respective second frame are different for different current frame (Toklu: 
column 11, lines 2-10). 

Re claim 15, Tolku discloses that said current frame, said first frame and said 
second frame are in consecutive successive positions in said sequence of frames 
(Toklu: column 5, line 65-column 6, line 7; column 1 1 , lines 2-1 0). 

Re claim 16, arguments analogous to those presented for claim 1 are applicable 
to claim 16, and, therefore, claim 16 has been analyzed and rejected with respect to 
claim 1 above. 

Claim 17 has been analyzed and rejected with respect to claim 2 above. 
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Re claim 23, arguments analogous to those presented for claim 1 are applicable 
to claim 23, and, therefore, claim 23 has been analyzed and rejected with respect to 
claim 1 above. 

Claim 24 has been analyzed and rejected with respect to claim 2 above. 

Re claim 29, Toklu discloses a method of identifying a plurality of key video 
frames in a sequence of image frames, each of said sequence of image frames 
containing a plurality of pixels, each of said plurality of pixels corresponding to a 
corresponding point of an area based on which said sequence of image frames are 
generated, said method comprising: receiving said sequence of frames of a same 
scene/area of interest according to a sequential order (Toklu: column 6, lines 30-32, 
several key frames may come from a single video segment); choosing one of said 
sequence of image frames as a current frame, a first flame being before said current 
flame and a second flame being before said first flame according to said sequential 
order, said first flame being at a first relative position in relation to said first flame in said 
sequential order and said second flame being at a second relative position in relation to 
said first flame in said sequential order (Toklu: column 5, line 65-column 6, line 7; 
column 1 1 , lines 2-1 0); calculating a difference of a first value and a second value, said 
first value representing a measure of a change of visual content of said current flame 
compared to said first frame, and said second value representing a measure of a 
change of visual content of said first flame compared to a second frame (Toklu: Fig. 2C, 
steps 228 and 229; Equation 1 1 , temporal color histogram activity curve used to 
generate binary color histogram activity curve); selecting said current flame as a 
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corresponding one of said plurality of key video frames if said difference exceeds a first 
threshold value and first value exceeds a second threshold value (Toklu: column 12, 
lines 37-42); and repeating said calculating and said selecting after choosing each of 
said sequence of image frames as said current frame to form said plurality of key video 
frames (Toklu: column 11, lines 2-10, calculated for all frames in a segment), wherein 
each of the respective first frame and the respective second frame are different for 
different ones of the current frames, and are respectively determined based on the 
same first relative position and said second relative position in reference to the 
corresponding current frame (Toklu: column 1 1 , lines 2-10, the process is iterative). 

1. Claims 3-4, 6, 9-11, 13, 18-19, 21, 25-26, and 28 are rejected Under 35 U.S.C. 
103(a) as being unpatentable over Tolku et al. (US 6549643 B1) in view of Zhang et 
al. (US 7027513 B2). 

Re claim 3, Toklu discloses a majority of the features of claim 3, as discussed 
above in claim 2 but Toklu does not explicitly state that said first representative 
magnitude for said current frame equals an average of motion energy vector 
magnitudes of said moved pixels of said current frame in comparison with 
corresponding pixels of said first frame. However, Zhang discloses a method for 
extracting key frames from video using a triangle model of motion based on perceived 
motion energy, where the average magnitude of the motion vectors for a particular 
frame is calculated (Zhang: column 9, lines 25-38), and the average magnitude is used 
to calculate the perceived motion energy (Zhang: column 9, lines 39-49). Since both 
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Tolku and Zhang relate to selecting key video frames based on motion analysis, one of 
ordinary skill in the art at the time of the invention would have found it obvious to 
combine the triangle method of Zhang with the histogram analysis of Tolku in order to 
provide a uniform and consistent selection method that yields a proper number of key 
frames that are most representative of the video sequence content (Zhang: column 5, 
lines 34-37). The combined method of Tolku and Zhang has all of the features of claim 
3. 

Re claim 4, the combined method of Tolku and Zhang discloses a majority of the 
features of claim 4, as discussed above in claim 3. Additionally, Toklu discloses that 
said first threshold value is the same for all of said current frames (Toklu: column 12, 
lines 16-19, the threshold is a constant in the preferred embodiment), said selecting 
further comprises: including said current frame is in said set of potential video frames 
only if said first representative magnitude exceeds a second threshold (Toklu: Fig. 2A, 
step 209 eliminates substantially similar frames; column 7, lines 56-65, elimination 
methods are shown in Figs. 3 and 4; Fig. 4, step 405, indicates a difference threshold); 
and including only those of said set of potential video frames, which exceed said first 
threshold, in said plurality of key video frames (Toklu: Fig. 4, steps 405 and 406, if the 
motion is not significant, the candidate frame is eliminated). 

Re claim 6, the combined method of Tolku and Zhang discloses identifying a 
plurality of active pixels in said current frame, wherein; a pixel is considered an active 
pixel if a corresponding displacement magnitude is outside of a range, wherein only said 
plurality of active pixels are used by said computing (Toklu: Fig. 2C, step 231). 
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Re claim 9, the combined method of Tolku and Zhang discloses enabling a user 
to specify one of a plurality of key video frames, wherein said plurality of key video 
frames are selected by said selecting (Zhang: Fig. 1, element 120; column 5, lines 43- 
44, indicate that the user may use the key frames to select the desired section of the 
video for display); and displaying said specified one of said plurality of key video frames 
(Zhang: Fig. 1 , element 1 20; column 5, -lines 43-44, indicate that the user may use the 
key frames to select the desired section of the video for display). 

Re claim 10, the combined method of Tolku and Zhang discloses displaying a 
prior key video frame and a next key video frame in relation to said specified one of said 
plurality of key video frames, wherein said prior key video frame and said next key video 
frame are comprised in said plurality of key video frames (Zhang: Fig. 1 , element 120, 
key frames are displayed in a sequence). 

Re claim 11, the combined method of Tolku and Zhang discloses generating a 
display indicating the manner in which said plurality of key video frames are 
interspersed in said sequence of image frames, wherein said enabling is based on said 
display (Zhang: Fig. 12, the key frames are shown along with their corresponding frame 
numbers). 

Re claim 13, the combined method of Tolku and Zhang discloses generating a 
display listing said plurality of key video frames, wherein said enabling is based on said 
display (Zhang: Fig. 1, element 120; column 5, lines 43-44). 

Claim 18 has been analyzed and rejected with respect to claim 3 above. 

Claim 19 has been analyzed and rejected with respect to claim 4 above. 
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Claim 21 has been analyzed and rejected with respect to claim 6 above. 
Claim 25 has been analyzed and rejected with respect to claim 3 above. 
Claim 26 has been analyzed and rejected with respect to claim 4 above. 

2. Claims 5, 7, 8, 20, 22, and 27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Tolku et al. (US 6549643 B1) in view of Zhang et al. (US 
7027513 B2) as applied to claims 3-4, 6, 9-11, 13, 18-19, 21, 25-26, and 28 above, 
and further in view of Ma et al. (US 20040088723 A1). 

Re claim 5, the combined method of Tolku and Zhang discloses a majority of the 
features of claim 5, as discussed above in claims 1-4, but does not specifically disclose 
that the first threshold and the second threshold are adjusted dynamically to ensure that 
a desired number of frames are selected as key video frames in a specified duration. 
However, Ma discloses a method for generating a video summary, where a binarization 
threshold is estimated in an adaptive manner (Ma: paragraph [0081]) when analyzing a 
video sequence for selecting key frames. Since Tolku, Zhang, and Ma relate to 
selecting key frames in a video sequence, one of ordinary skill in the art at the time of 
the invention would have found it obvious to combine the attention modeling of Ma with 
the combined key frame selection method of Tolku and Zhang in order to create a 
representative video summary consisting of very short video clips, which contain the 
video immediately preceding and immediately following key frames that have been 
selected (Ma: Fig. 20). The combined method of Tolku, Zhang, and Ma has all of the 
features of claim 5. 



Application/Control Number: 1 0/708,905 Page 1 1 

Art Unit: 2621 

Re claim 7, the combined method of Tolku, Zhang, and Ma discloses that said 
range set by a distance of two times the variance from the mean of a distribution (Ma: 
paragraphs [0081]-[0082], the threshold is a function of the variance). 

Re claim 8, the combined method of Tolku, Zhang, and Ma discloses that said 
representative magnitude comprises an average of said active pixels (Ma: equation (12) 
and paragraph [0084]). 

Claim 20 has been analyzed and rejected with respect to claim 5 above. 

Claim 22 has been analyzed and rejected with respect to claim 7 above. 

Claim 27 has been analyzed .and rejected with respect to claim 5 above. 

3. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Tolku et al. (US 6549643 B1) in view of Zhang et al. (US 7027513 B2) as applied to 
claims 3-4, 6, 9-11, 13, 18-19, 21, 25-26, and 28 above, and further in view of Sull et 
al. (US 20060064716 A1). 

Re claim 12, the combined method of Tolku and Zhang discloses a majority of 
the features of claim 1 2, as discussed above in claims 1 -4, 6, and 9-1 1 , but does not 
specifically disclose that the display comprises a pie chart. However, Sull discloses 
techniques for navigating multiple video streams, where textual/visual information, such 
as a pie chart, may be displayed along with poster-thumbnails on the video selection 
screen for the user interface (Sull: paragraph [0307]). Since Tolku, Zhang, and Sull all 
relate to representative images for video sequences, one of ordinary skill in the art at 
the time of the invention would have found it obvious to combine the textual information 
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of Sull with the keyframe selection of the combined method of Tolku and Zhang in order 
to provide the user with more information, such as date and time of broadcast (Sull: 
paragraph [0307]), for improving the user's ability to quickly find the desired video 
segment (Zhang: column 5, lines 44-45). The combined method of Tolku, Zhang, and 
Sull has all of the features of claim 1 2. 

Conclusion 

4. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

a. Feature based hierarchical video segmentation; Bozdagi et al. (US 
6493042 B1 ) 

b. Method of selecting key-frames from a video sequence; Wilf et al. (US 
7184100 B1) 

c. System for automatic video segmentation and key frame extraction for 
video sequences having both sharp and gradual transitions; Zhang et al. (US 
5635982 A) 

Contact 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTOPHER FINDLEY whose telephone number is 
(571 )270-1 1 99. The examiner can normally be reached on Monday-Friday (8:30 AM- 
5:00 PM). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha D. Banks-Harold can be reached on 571-272-7905. The fax phone 
number for the organization where this application or proceeding is assigned is 571 - 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Marsha D. Banks-Harold/ 

Supervisory Patent Examiner, Art Unit 2621 

/Christopher Findley/ 



